Multimodal Tucker Decomposition for Gated RBM Inference

Gated networks are that contain gating connections in which the output of at least two neurons multiplied. The basic idea a gated restricted Boltzmann machine (RBM) model is to use binary hidden units learn conditional distribution one image (the output) given another input). This allows RBM transformations between successive images. Inference consists extracting pair However, fully connected multiplicative network creates cubically many parameters, forming three-dimensional interaction tensor requires lot memory and computations for inference training. In this paper, we parameterize bilinear interactions through multimodal tensor-based Tucker decomposition. decomposition decomposes into set matrices (usually smaller) core tensor. parameterization helps reduce number reduces computational costs learning process effectively strengthens structured feature learning. When trained on affine still images, show how completely unsupervised learns explicit encodings transformations.